Performance of a trialogue-based prototype system for English language assessment for young learners

This paper describes a trialogue-based system for assessing the spoken language abilities of young learners of English. Specifically, the system employs spoken dialogue system components in interactive, conversation-based assessment tasks involving the test taker and two virtual interlocutors. The tasks are designed to be engaging for young learners of English at the elementary school level by incorporating real-life situations into the conversations and by providing immediate feedback about their spoken responses. The system was deployed in a data collection experiment with 18 young learners of English in a public school in the USA (grades 3 5) from a variety of language backgrounds. The system was able to produce overall task completion scores for each participant that correlated with scores based on human annotations at a rate of r = 0.803. In addition, the participants’ responses to a user experience survey indicate that most participants felt that the virtual interlocutors understood their responses. This prototype demonstrates that this approach to using interactive, conversation-based assessments is a viable method of assessing the English language skills of young learners.